Variable-Span out-of-vocabulary named entity detection

Out-of-vocabulary named entities (OOV NEs) are always misrecognized by fixed-vocabulary automatic speech recognition (ASR) systems. This has a negative impact on downstream applications such as language understanding and machine translation (MT). Automatic detection of OOV NEs in ASR hypotheses can help mitigate this problem by triggering the use of alternative approaches to acquire and process these NEs. State-of-the-art OOV NE detection typically involves tagging ASR-hypothesized words using a sequence model, such as conditional random fields (CRF), in conjunction with a variety of contextual and ASR-derived features. In this paper, we propose a novel variable-span tagging approach for detecting OOV NEs. Instead of tagging individual words in ASR hypotheses, we directly tag longer spans of consecutive words. The proposed approach outperforms a state-of-the-art CRF tagger on two distinct heldout test sets with different OOV NE distributions. On a 5.1Kword test set rich in OOV NEs, our method achieves 56.1% detection rate at 10% false alarm rate (vs. 52.1% for the CRF detector). On a 39.4K-word test set with a natural distribution of OOV NEs, we obtain 73.0% detection rate at 10% false alarm rate (vs. 69.5% for the CRF detector). In all cases, OOV NEs are completely unobserved in our training data.